The invention relates to publish/subscribe messaging broadly, and more particularly relates to a novel system and method for publish/subscribe messaging that maintains pre-emptive subscriptions to “subscribed to” topics without the use of cookies to reduce numbers of individual subscriptions managed by a message queuing function from “one client to one subscription” to “one unique subscription to one HTTP server,” obviating any need for establishing “client-affinity” between a client and any particular HTTP server.
Publish/subscribe is an asynchronous messaging technology whereby senders (publishers of messages) do not send or publish messages to specific receivers (subscribers of messages), but to topics, or named logical channels. Messages are collections of binary or character data that have some meaning to a participating program. Publish/subscribe data processing systems have become very popular in recent years for distributing data messages (publications between application programs). Publishers are typically not concerned with where their publications are going, and subscribers are typically not interested in where the messages they receive have come from. Published messages are characterized into classes without knowledge or regard to which subscribers (if any) should receive the messages. Subscribers typically receive only a subset of total messages published. The publisher is responsible for defining the classes of messages to which subscribers can subscribe.
Publish/subscribe systems may be either topic or content based. In a topic-based system, messages are published to topics or channels associated by topic. In a content-based system, messages are delivered only to subscribers where attributes or message content matches constraints defined by the subscriber, who is responsible for classifying the messages. A message broker typically assures the integrity of the message source, and manages the distribution of messages according to valid subscriptions registered with the message broker. A configuration manager drives deployment, including storing and managing configuration data used to configure brokers, and hosted by the message queue manager.
Publish/subscribe systems evolved from WebSphere MQ™, where the WebSphere MQ platform now implements an HTTP interface. As in other communication protocols, storage, routing, and delivery information is added to the message before transmission, and stripped from the message prior to delivery to the receiving application. Message queues are objects that store messages in an application. A queue Manager provides a logical container for the message queue. The queue manager (or message broker) is responsible for transferring data to other queue managers via message channels.
Messages are not dependent upon pure packet-based transmissions, such as TCP/IP, to implement the sending (publisher) and receiving (subscriber) ends to be decoupled, and potentially operate asynchronously, thereby. Messages are delivered once and once only, irrespective of errors and network problems. Use of the HTTP interface allows the user to POST and GET messages from either a topic or queue. In known systems, the initial implementation of receiving messages from a topic had a “window of opportunity” whereby clients could “miss” messages being published on the topic. Several implementation scenarios are outlined below to highlight some shortcomings of conventional techniques.
Scenario 1
1) Client requests message from topic.
2) Server waits a given period to see if any messages are published to that topic.
3) Server returns a message or a “no message received” response.
4) Client reconnects in order to receive next message.
One problem with this “window of opportunity” implementation (scenario), however, is that between actions 3 and 4, a message could have been published on the topic but the client would not receive it. This “window of opportunity” implementation does not cater for any kind of subscription. The following second implementation (scenario) reflects an improvement of the “window of opportunity” implementation.
Scenario 2
1) Client subscribes to topic
2) Client returns to server asking for any messages that have been received since subscribing.
3) Server returns a message or a “no message received” response.
4) go to 2 until such time as messages are not required.
5) Client unsubscribes from topic.
The second scenario, however, highlights a second limitation of known publish/subscribe system operation. That is, for each client that wishes to listen on a topic, a separate subscription is created. In a web-based environment, many tens (and potentially hundreds) of thousands of users could be connected at once. Such a connection obviously requires an enormous number of connections between the HTTP server and the Message queuing system where a 1:1 affinity is maintained between clients and subscribers (subscriptions). But in addition to requiring a great number of connections, where more than one HTTP server takes part in the transaction, the implementation scenario breaks down.
As most HTTP environments rely on horizontal scalability (more than one HTTP server in the cluster), the HTTP-based system must be constructed with an ability to adequately respond to a client potentially being directed to differing HTTP servers on each subsequent request for messages. For that matter, message selection is also an issue in the second scenario. As most messaging technologies already have message selection pushed down into the messaging server, it is important that any solution does not attempt to reproduce message-matching algorithms outside of the messaging system.
Besides the WebSphere MQ publish/subscribe platform with HTTP interface, other known publish/subscribe technologies have similar problems. For example, Bayeux's known publish/subscribe technology platform is faced with the same problems. Bayeux, however, has not implemented a solution that is truly horizontally scalable but instead relies on a client remaining at a single HTTP server using “HTTP keep-alive.” Hence, if a client becomes disconnected, they must reconnect using the initial handshaking protocols (where the subscription is re-created when messaging using a Bayeux platform). Bayeux implements its publish/subscribe system with message matching carried out by/within the HTTP server code, rather than by the message engine code.
In addition to use of “HTTP keep alive,” Bayeux also implements what are known as “durable subscriptions.” Durable subscriptions comprise known publish/subscribe methodology whereby a unique client ID is sent by the client on each message request. Individual servers, therefore, by access to this available unique client ID, have no need to retain client information as to which message the client has received. But this solution, however, is not without shortcomings. For example, there is a one to one mapping between client subscription and server subscription, which is very resource intensive at the messaging server.
The WebSphere MQ platform with HTTP interface implements durable subscriptions. Consequently, if a message is published to a durable subscriber, the subscriber's message copy is maintained within MQ as persistent, regardless of whether the message was published as a persistent message. The problem with such operation is that it is performance prohibitive because all messages then require serialization to disk. Serializing all messages to disk typically results in a potentially large drop in performance, cluster-wide.
A system and method according to inventive principles addresses these deficiencies and related problems.